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ABSTRACT 

A brief review of the papers presented at the Symposium on 
Research and Evaluation Methodology highlights some of the major issues of 
research in Indian Country. There is a need to translate research results 
into terms that are meaningful at the local level, yet the act of doing this 
reveals tensions between scientific objectivity and advocacy. Tribes have 
developed research ethics codes, but recognition of them by mainstream 
institutions has been slow. Sampling issues reveal a tension between 
generalizability and local specificity of findings, which have different 
kinds of constituents and different lessons attached to them. The imposition 
of national evaluation models also involves the tension between the general 
and the specific. The development of tribally operated and administered 
institutional review boards presents numerous challenges, not the least of 
which is how to fund them. There are issues involved with confidentiality. 
Sponsoring agencies often want all data made public, but tribes may insist on 
community confidentiality. Using Native persons for data collection is often 
assumed to enhance rapport between researchers and study participants, but 
there are disadvantages to be considered. Issues involved with informed 
consent include language barriers, cultural differences, and who is 
authorized to give informed consent. Spirituality is important to Native 
communities, yet measuring it is problematic, especially with Western 
scientific methods. The length of time involved in conducting culturally 
respectful participatory research presents its own barriers to young 
researchers whose success depends on publication. (TD) 
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I 've organized my remarks into two parts. 
First, I have a series of general reactions to 
each of the individual papers, which I'll take 
in the order of their presentation. Second, in 
synthesizing the lessons learned that you've 
shared in your respective contributions and 
considering them in the context of my own 
experiences, I have noted a variety of 
challenges and opportunities that are reflected 
in this work. I'll attempt to summarize them 
succinctly, as a platform for further discussion. 
By no means is this meant to be an exhaustive 
set of comments; rather, I've chosen to 
highlight a few points and to seek clarification 
on others. 

Beginning with A Brief History of and Future 
Considerations for Research in American 
Indian and Alaska Native Communities — it's a 
really nice piece, Jamie. It provides the 
general historical context that any reader — 
investigator, sponsor, or consumer — in this 
particular area needs as the basis for 
understanding the challenges before us and the 
backdrop to those challenges. 

Early on in your paper, Jamie, you note the 
importance of translating the results from our 
work as investigators, or from the scientific 
research process, into terms that are 
meaningful at the local level, to the key 
stakeholders (to borrow Walter's term). This is 
critically important for several reasons. One is 
that most of us, based upon the nature of our 
training, are not adequately prepared to speak 
in terms that are easily and appropriately 
consumed by this set of constituents. This 
poses a major problem for us; I know that it 
does, in my experience. In the context of tribal 
review, we prepare scientific manuscripts for 
which we're seeking the tribe's comments; we 
also provide community summaries of no 
more than two pages double-spaced. Nothing 
drives home the challenge in this translation 
process more than something as mundane as 



using the Microsoft Word grammar-level 
function to figure out the grammar level we've 
used and the language complexity we're 
employing. We always exceed the 12th grade 
level in the first try, and work very hard to 
simplify it to 8th and 9th grade level. I don't 
say this pejoratively, implying that our 
constituents are simple or that their language 
abilities are limited. The challenge is to render 
it understandable to most people! I use my 
parents as the model, because my parents 
understand both the content and the import of 
this work. If I can explain it to them, then I'm 
likely to reach most audiences. This is no easy 
task; it's one we're not typically trained to do, 
but it is critically important. 

Jamie, the other issue that you've introduced to 
us — and I'll return to it in my general 
remarks — is the tension that's at work here. 
One of the other papers talked about it in terms 
of the subjective-objective relationship, that 
we as investigators working in this particular 
field find ourselves asking, "Where does 
science end and advocacy begin?" Jamie, you 
talk about the importance of translating the 
results of this kind of work into appropriate and 
meaningful local applications. I agree entirely 
with you. But I also submit that our training 
does not equip us to undertake this particular 
set of tasks. 

So how do we go about doing this? Gatherings 
such as this are an important part of the 
process. They establish role models. Virtually 
all of us do this kind of translation. But we 
have to be able to demonstrate it for our 
younger colleagues, to illustrate how it's 
appropriately done. We have to be able to 
share with them the angst, the trials, and the 
rewards that accompany this struggle. So this 
is a particularly important message that you 
conveyed, Jamie. 

As you reviewed the historical backdrop, 
although his name doesn't appear, his voice 
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certainly does: Vine Deloria. All of us need to 
remember Vine Deloria's important work from 
the early '60s through Red Earth , White Lies in 
1995, in which he talks about the way local 
knowledge in our respective Native 
communities has been judged inferior to the 
assumed eminence of western science. This is 
one of your messages: colonization takes a 
variety of different forms and its effects last 
many decades. 

Jamie, as well, the last portion of this particular 
manuscript talks about best practices. You 
provide the readership a very nice overview in 
regard to the different types of research ethics 
codes that are emerging in Indian Country, and 
that are even now finding their way into such 
taxonomies as the Diagnostic and Statistical 
Manual of Mental Disorders (American 
Psychiatric Association, 2000). As a 
psychologist, you are particularly interested in 
the latter; it's probably one of the reasons you 
chose it as an example. But this example 
illustrates the ways in which some doors are 
opening even in the most conservative 
institutions — psychiatry among them. And, 
Jamie, I encourage you to review Culture, 
Medicine and Psychiatry, beginning with 
Candace Fleming's article (1996) that helped 
establish the case studies series in this 
publication. Three of us have published case 
studies in this forum, all involving American 
Indians, using the DSM-IV diagnostic 
formulation for cultural outline. Candace, 
Terry O'Neil, and I employed American Indian 
cases to illustrate how this kind of a cultural 
formulation moves us well beyond the typical 
gains of business as usual. 

So, a very nice piece. Thank you. 

Regarding Research in Indian Country: 
Challenges and Changes, Walter, always a 
delight to read your work. It's very important 
that you have called our attention to these 
three roles — sponsors, consumers, and 
stakeholders — and critically important for us, 
as scientists working through such 
partnerships, to translate the outcomes of these 
efforts into meaningful terms across all three 



categories. This tripartite distinction is really at 
the heart of what may be our first step: a 
critical analysis of the ways in which these 
kinds of relationships should unfold and what 
kind of work we can best apply ourselves to. 

Walter, you made a very nice point in regard to 
the role and responsibility of federal agencies' 
sponsorship of research in Indian Country. We 
really do owe a great deal to the Clinton 
Administration with respect to its support of 
and advocacy for government-government 
relationships and consultations. We're 
beginning to see this come to fruition in 
various agencies, but not as aggressively nor as 
comprehensively as such consultation should 
be. The extent to which this is seriously 
addressed will take us much further in terms of 
these sponsors understanding the kinds of 
issues that are before , us. Let me give an 
example. One of the new, major initiatives 
that the National Institutes of Health (NIH) is 
required to address has to do with public 
access to data generated by its grants. Now, 
this poses a very particular set of concerns that 
are thorny for us working in Indian Country, as 
we consider the full ramifications of such 
demands. This requirement poses often 
competing demands when we work with our 
tribal and community-based partners. 

Walter, you hit the nail on the head in your 
manuscript when you wrote about the major 
scientific challenges that face us with respect 
to sampling: the "denominator problem," 
whether it be with the census or with the 
community-based work that we must do to 
capture the data needed to address broader 
issues. In our work, epidemiologically, etc., 
sampling is an enormous challenge for us. 
And it brings us back to an issue that I'll return 
to in my more general remarks: the tension 
between generalizability and local specificity 
of research findings, which have different kinds 
of constituents and different lessons attached 
to them. 

Walter, you also talk about there being an 
increasing number of institutional review 
boards (IRBs) in Indian and Native 
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communities. Through a. fairly systematic 
survey I'm aware, right now, of nine formally 
chartered IRBs in Indian Country, three or four 
of which are very active. However, it's not my 
sense that at present many IRBs are tribal ly 
operated and administered. For the most part, 
many tribes look to the Indian Health Service 
and its area offices to serve that function for 
them. I'll return to this point. It's a big issue 
for our communities, for us, and for sponsors. 
How does one promote, support, and 
encourage the development of tribal-level 
IRBs? There are a number of challenges for 
everybody; not the least is how they'll be 
financially supported, which is not clear — and 
often not even the case. Our investigative 
team has discovered several solutions, but 
until our sponsors recognize that part of their 
responsibility is to help finance such review 
structures, we will be frustrated by the speed 
with which we see tribal IRBs working in our 
respective communities. 

Walter, you raise several interesting tensions 
having to do with the issue of confidentiality. 
I'm always amazed at the extent to which non- 
Native colleagues are surprised by the 
emphasis on community confidentiality in 
reporting research findings. They understand 
and respect the importance of confidentiality 
regarding the individual, but are puzzled by 
why many communities would want this 
respect extended to their entire communities. I 
refer them to an article published some years 
ago that led to the establishment of the Navajo 
Nation Health Research Review Board. For 
that article, CDC investigators, in collaboration 
with the Navajo Health Authority, had agreed 
to present their manuscript on the 
epidemiology of the Hanta Virus during the 
initial peak of concern. The Navajo Nation 
approved it, asking that the 20 communities 
not be identified but named in a way that 
wouldn't diminish the scientific basis of the 
findings, thereby avoiding local stigmatization. 
The CDC agreed — but when the article 
appeared the Navajo communities were 
named, abrogating their agreement. Early in 
that investigation, a newspaper in Denver 



carried a front page headline, a half-page 
picture of a hogan, and a caption that spoke of 
the "Navajo flu" and dirty living conditions, 
implying that traditional lifestyle contributed to 
the risk of that particular disease. This was 
right on the heels of that particular publication 
by the CDC — underscoring the stigma and the 
rightful concerns of the Navajo Nation. 

Walter, you talk about how the quality of the 
data collected is often improved by enhanced 
communication, better rapport between 
researchers and study participants. Then you 
qualify this as you talk about the costs and 
benefits of trying to accomplish this match 
between the people who collect data and the 
people who give it as a gift. It's a very 
important point. For the most part, those who 
are inexperienced or less experienced with 
these issues assume that it's a ready-made 
solution: if a Native person is part of your data 
collection team, you will be immediately 
catapulted over such problems. You provided 
several excellent examples to the contrary, 
illustrating how it's not necessarily the 
solution, the panacea, one might assume. 

In a recently completed two-stage epi- 
demiological study of alcohol use and 
dependence in a Northern Plains community, 
the first stage involved administering sensitive 
questions to community members. The 

reported rates of alcohol abuse and 

dependence were quite low in the data 
generated by that phase. But in the second 
phase, a clinical reinterview of a large 
subsample of the same study participants, we 
observed rates of alcohol abuse and 

dependence several times greater than 

estimates based on the first stage. In debriefing 
our team we found, in fact, that we had 
inadvertently staffed one of our field offices 
largely with recovering alcoholics — so the 
community members whom they interviewed 
in the first stage felt less disposed to disclose 
their true personal histories of alcoholism. 
This reminds us that there are trade-offs. The 
message you've conveyed is that we need to 
be critically thoughtful about the advantages 
and disadvantages in hiring Native persons 
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from a given community to assist with data 
collection in that community. 

Walter, you also talk about the issue of 
individual informed consent. There are two 
excellent examples of the challenges that we 
face in Native communities as we approach 
this issue. Joseph Carrese, an associate editor 
of the Journal of General Internal Medicine at 
Johns Hopkins, published an article about six 
years ago (Carrese, 1 995) that demonstrates so 
powerfully the challenges that we face in 
appropriately informing study participants, and 
the role that culture plays in this regard. The 
example he gives is that among the Navajo it is 
believed that giving voice to bad things, 
negative things, increases the probability that 
those things will occur. 

What do we do in an informed consent form? 
We present a long litany of possible risks. We 
characterize all the possible adverse events 
that might occur through participation in this 
study — hence violating at the very outset the 
Navajo view that giving voice to such things 
increases the likelihood of their occurrence. 
How do we address such issues in the 
informed consent process? And that's beyond 
all of the other potential barriers, such as 
language comprehension, etc. 

There's another issue with respect to informed 
consent that we experienced (Norton & 
Manson, 1996); it has to do with who is 
appropriate to give informed consent. In this 
particular study we worked in five different 
tribal culture areas; one was in the southwest, 
involving several pueblos. In visiting the tribal 
council, we reviewed the research protocol, 
indicating that individual informed consent 
would be required. The council refused. They 
said, in effect, "We as the Tribal Council are 
responsible for providing consent for all of our 
tribal members." It was an expression of the 
manner in which that tribe is organized 
socially, in terms of its collective responsibility 
for members. 

This proved to be an extremely thorny issue 
with our institutional review board (IRB) here 
at the University of Colorado Health Sciences 



Center. Ultimately, a compromise was 
reached. But this example underscores how 
even the notions that guide the informed 
consent process and that are embodied within 
our IRB processes reflect particular cultural 
assumptions about responsibility, authority, 
and individuality. Walter, those strike me as 
two useful examples that you might use to 
emphasize this point. 

Regarding Cultural Competence Approaches 
to Evaluation in Tribal Communities, Paulette, 
delightful; a pleasure to read your paper. 
You've nicely captured an issue that Catherine 
and her colleagues note in their manuscript as 
well: the importance of attending to the local 
construction of the phenomena, the 
experiences, that our research seeks to 
understand, describe, and explain. Your 
example, Paulette, relates to the concept of 
disability. You mention it in the context of the 
Global Assessment of Functioning (GAF) 
(American Psychiatric Association, 2000) and 
of various tribal grantees' attempts to 
operationalize this construct in locally 
meaningful terms. This is a key struggle for us, 
and for sponsors as well, to recognize. This is 
an instructive lesson, Paulette, and there are 
other examples in the literature which you 
might consider referencing here. Especially 
when coupled with the manuscript by 
Catherine and her colleagues, your paper 
exhorts us to be more reflective, more 
systematic, in our search for locally meaningful 
constructs of this nature. 

A current example, not reflected in these 
manuscripts but certainly anticipated by them, 
is "spirituality." Paulette, your manuscript 
touches a little bit on spirituality; our Circles of 
Care initiative is centrally concerned with this 
concept. How do we measure spirituality? 
How do we build it in to our evaluation 
strategies? Into our research approaches? We 
were driven to do this in our work. I was 
reluctant, as a scientist, because it seemed to 
threaten polluting the sacred with the profane. 
By that I mean, why should we assume that 
western scientific methods lend themselves to 
measuring such a sensitive construct as 
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spirituality — or that it would even be 
appropriate to do so? Frankly, it was our 
Navajo colleagues who challenged me on this. 
They said, "We want to. To the extent that it 
can be measured, we believe it will prove 
critically important to understanding risk and 
protection among our people, as well as 
broadening the theories and models available 
to us from the general literature." So that is a 
lovely point. 

The converse, which you also talk about, is the 
frequent observation that "wraparound 
services" is a familiar notion in Indian 
communities, one we've adopted for a long 
time. It's probably no accident that John 
VanDenBerg (VanDenBerg & Minton, 1987), 
one of the earliest advocates of wraparound 
services, did his first work in Alaska. Much of 
his work in the Alaska Youth Initiative involved 
Native villages, wherein life emulates many of 
the basic tenets of the wraparound model. So 
another important observation is not just that 
it's important to adapt these kinds of things for 
application in Native communities, but that 
there are lessons to be learned from doing so, 
and that the greater world of science can 
benefit from careful attention to these matters. 

Now I'm going to take you to task a little bit. I 
can do so, being a little more privy than most 
to some of the tensions that emerged with 
respect to the Systems of Care initiative 
through my participation at meetings that 
brought the Indian and Native grantees 
together with ORC Macro to debate the 
adequacy and the felt imposition of the 
national evaluation model on the formers' 
communities. You persuasively characterized 
the way in which the Indian and Native 
grantees challenged ORC Macro to develop 
new constructs or to adapt the national 
evaluation plan's existing constructs. But 
what's lost in your discussion, Paulette, and is 
important for the audience to share in, is the 
tensions that were at issue, not just that they 
were resolved. 

That will return us later to more general 
remarks regarding tension between the general 



and the specific. Meanwhile, though, it would 
be helpful if you elaborated and shared the 
tensions and the demands on Macro. They 
had a responsibility to develop a national 
evaluation plan, and did so in good spirit and 
with good intention based upon a particular set 
of scientific tenets about the notion of 
comparability, reinforced by a scientific 
advisory board — a board that I happen to be 
on. Yet, at the same time, Macro was 
pressured by Congress and by the Center for 
Mental Health Services (CMHS) to produce 
evidence as to the effectiveness of this 
particular initiative that cuts across the 
grantees. Grantees sought to be responsive to 
these demands, recognizing their contractual 
obligations. But, at the same time, they 
suffered from the imposition of an external 
view of process and outcomes. At the same 
time, the grantees did not understand the 
organizational constraints on Macro that limit 
its ability to respond. So, it would be helpful 
to the audience if you bring to the fore a bit 
more the human fabric that underlies some of 
these tensions. One of the lessons here is 
demonstrating or chronicling these lessons, 
enabling other agencies, other sponsors, and 
other grantees to acquire a better sense of what 
issues to anticipate and the paths to their 
solution. Of course, they're not yet solved at 
this point, but well on their way to appropriate 
solutions. A lot of this is, as you know, process. 
This is the piece missing here: conveying to the 
audience the process by which these important 
milestones were achieved. 

Okay, Paulette, now I'll let you off the hook; 
it's a nice piece of work. 

Catherine, I was very pleased to see that 
Community-Based Research and American 
Indians with Disabilities: Learning Together 
Methods that Work, more than any other paper 
here, takes head-on the issue that "culture 
counts" — a phrase borrowed from the APA 
Monitor summary (Daw, 2001), as cited in 
your paper. In fact, there's a whole chapter on 
culture that appears in the 2001 supplement to 
the Surgeon General's 1999 report on the 
Mental Health of the Nation (U.S. Department 



0 



BEST COPY AVAILABLE 



85 



of Health and Human Services, 1999). It was 
a major battle to get the various agencies to 
actually adopt this language — that "culture 
counts." Your paper, more than any other 
here, underscores the issue, and reminds the 
audience that this is not just a matter of 
American Indians or Alaska Natives vis-a-vis 
some broader societal group, but a matter of 
culture, involving process, dynamic, meaning. 
So this is critically important. 

This manuscript is also the only one that talks 
about urban issues, which are vitally 
important. You speak about the strengths and 
weaknesses of your community assessment 
approach, which was pencil-and-paper 
originally, and your subsequent adaptation of 
the method. You did this in Denver and Texas, 
if memory serves, and subsequently in 
Minneapolis. It's a nice example of how to 
carry forward a set of methods, adapting to the 
voices in our respective communities. 

In this portion of your work you're drawing 
upon your own experience, but there's another 
example that may be useful to reference, 
having been used both in reservations as well 
as urban settings: the Community Readiness 
Model developed by Colorado State University 
(Jumper-Thurman, Beauvais, Plested, Edwards, 
Helm, & Oetting, 2001). It serves a similar 
kind of purpose. We all would benefit by 
looking more closely at both kinds of 
approaches. Each represents thoughtful, 
systematic ways of giving voice to the 
stakeholders, in Walter's terms, who are 
represented in these communities. 

This manuscript goes to great lengths about the 
importance of key collaborators. I fully agree 
with the underscoring of the importance of the 
key collaborators. But let me note that the 
people who choose to collaborate with us are 
not devoid of their own agendas. They have 
their particular views. They have their 
particular priorities. It's important for us to be 
critical, analytic, about who chooses to 
collaborate, what may be at stake for them, 
and what they see as the issues, benefits, and 
costs. Collaboration is critical. Identifying key 



stakeholders to participate in these 
partnerships is critical. But another important 
part of the process is to retain a self-reflexive 
perspective, to understand who has agreed to 
collaborate with us and whom we have 
collaborated with. We all know that in 
working in our communities, joining with one 
person often precludes our ability to work with 
others. And what does that have to say about 
the particular views to which we are privy? 
About the tasks we have chosen to undertake? 
About the outcomes desired and thought 
possible? 

Learning from and Working with Yup'ik 
Professionals is a lovely work. As you noted, 
Jennifer, it is different from the others in terms 
of initial orientation and focus. But it's an 
extremely valuable contribution. You've taken 
the Ages and Stages modules and demon- 
strated the use of certain organizational 
techniques to build teams with certain kinds of 
outcomes. You've taken us more deeply into 
the processes often at work at the more 
abstract level. You've done it within a 
systematic framework that allows us to 
examine how those processes unfold in similar 
or different ways among Yup'ik and non- 
Native northwest professionals. Although I 
encourage you to explore this even more 
deeply, you were able to talk about some of the 
apparent "disconnect" between the Yup'ik 
approach and how their cultural values of 
respectful listening and patience seemed at 
some points to run counter to the process. In 
debriefing Yup'ik participants, you asked what 
they might like to change in a reiteration. They 
suggested more structured settings that would 
allow them to comfortably adopt the particular 
values assumed in the Ages and Stages 
approach, thereby enabling them to engage in 
a dialogue, in a narrative, in discussions that 
would otherwise be more difficult. But this is 
really more than a case study, Jennifer. 
Quantitative and qualitative investigative 
techniques have been integrated and then 
applied to a process that may be generalizable 
beyond these particular groups. Let's face it, 
issues of organizational culture are among the 
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biggest challenges for us to address as we 
navigate the various partnerships that emerge 
in our work. 

Now, let me try to characterize for you more 
broadly what I believe to be the challenges as 
well as the opportunities that your papers 
anticipate for us, and that my modest 
experience suggests to me, as well. 

First, the tensions anticipated in my earlier 
comments. There is a general tension between 
science and advocacy that we need to address 
openly. It goes to the heart of the manner in 
which we're trained and how most people 
construe the appropriate role of science and 
scientist. Only recently have sponsors such as 
NIH begun to talk about the importance of the 
translation process. They're even talking about 
"translational science" — about how to 

systematically translate research findings, 
whether in the lab or more controlled 
experimental settings, into meaningful, real- 
world applications. We're not trained to do 
this; we're not well equipped to make such 
translations. These efforts also typically are not 
funded by our sponsors, so that it falls to us to 
expend the extra time and energy to 
accomplish them. The translation process also 
engenders a fair bit of discomfort for us; we're 
extending ourselves into areas that are less 
familiar than other areas. But this is an 
important undertaking; our communities 
demand this of us, that we engage in advocacy 
and in this translation process. 

There are other tensions mentioned earlier, and 
that Walter touched upon in his discussion of 
sampling: generalizability and local specificity. 
Paulette, it surfaces among the evaluation 
issues that you and Brigitte face with respect to 
Macro. How does one deal, scientifically, with 
the notion of comparability in a way that is 
locally meaningful and that speaks to the 
integrity of the initiative you're a part of? How 
do we generate outcomes or results that can 
speak to the general effort, but that are at the 
same time locally meaningful? Not easy tasks. 



Yet a different tension springs from the 
mandate to compare — namely, the assumption 
that uniformity somehow brings with it, as a 
natural consequence, comparability. Not 
t rU e — but it's part of the challenge that we face 
as scientists, in educating our fellow 
colleagues as well as in working in our 
respective communities, to find points of 
compromise. 

There are other tensions as well, between 
academic freedom and tribal sovereignty. 
Working recently with a colleague, I became 
distressed at how much more work we have 
yet to do and progress yet to accomplish. The 
individual in question had conducted work in 
a community with which we had agreed to 
present all presentations and publications for 
prior review and approval. Despite being 
Native, despite being aware of our agreement, 
this colleague, wishing to present a paper at a 
forthcoming conference, but absent tribal 
approval, asserted the primacy of academic 
freedom over tribal sovereignty, couching the 
issue in terms of censorship. There may be 
some merit to her argument, rooted in the 
history of how we've been trained to do 
science and the expectations of the institutions 
in which we work. But it was appalling to me 
that this argument issued from a young, Native 
colleague, underscoring the work still before 
us. 

Shifting from these tensions to other 
challenges, I mentioned earlier the issue of 
supporting — not just in principle, but 
financially — local tribal review processes. This 
is not a straightforward matter. Our experience 
over the last four years of working directly with 
tribal IRBs reveals an enormous amount of 
variability among them in terms of their 
organization, the speed at which they move, 
the kinds of information they request, their 
decision-making processes. And not minimal 
by any means is the issue of how that entire 
endeavor is financed at the local level. If 
we're serious as sponsors, consumers, and 
stakeholders about promoting tribal review, we 
have to become serious about funding it. 
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Discussions with the National Institute of 
Mental Health (NIMH) in recent months have 
yielded their agreement to include in grants 
subcontracts to our tribal partners to fund this 
process. Yet, several other institutes have 
rejected this approach out of hand. There is 
considerable variability among the institutes at 
NIH, but this foothold gives me greater 
optimism for addressing this important need. 

I've also suggested to several of these tribal 
review boards, each dramatically under- 
funded, that they charge for the review 
process, which also makes it easier to build 
such costs into research funding mechanisms. 
In Colorado (and this is true in many 
other universities), the Colorado Multiple 
Institutional Review Board charges $1,200 to 
review applications not submitted by members 
of our institution. Well, why can't tribal IRBs 
adopt a similar private enterprise model for 
those investigators wishing to work in their 
communities? There are solutions out there. 
We just have to become more aggressive in 
seeking them. 

One issue that arises as a natural segue with 
respect to this issue of local review has to do 
with ownership of data. I confess to being 
caught in the middle of interesting struggles 
about the ownership of data. NIH, the sponsor 
of most of our work, asserts that we, as 
principal investigators, own the data. My 
institution has a particular view on that as well. 
Several of the tribes with which we work also 
assert that they own the data. So what does 
ownership mean? Is there multiple ownership 
of data? Are there different but complementary 
applications of the data? How does one 
pursue this notion of multiplicity of ownership? 
How does one negotiate the respective 
responsibilities and areas of mutual 
responsibility? 

NIH has instituted a major policy regarding 
public access to data to ensure that data is 
shared, that it doesn't just sit in some 
investigator's computer unused for decades, 
that the investment the American public has 
made in sponsoring such work actually sees 



the light of day. How does one think about 
meeting the spirit of this intent when working 
with entities that also claim ownership of the 
data? And we all have very appropriate 
concerns about what is subsequently done 
with data. 

Walter, you pointed out that the greatest area 
of contention right now is in terms of the 
ethical, social, and legal implications of 
genetic research, particularly in American 
Indian and Alaska Native communities. That 
particularly volatile issue involves com- 
mercialization and the subsequent profit that 
may derive from gene banks, etc. These recent 
concerns heighten the debate, and tend to 
polarize views on the research process. How 
do we navigate these shoals, remaining open 
to reason and reasonable arguments? 

What's the threshold for seeking permission to 
disseminate findings? Is it simply naming our 
tribal partners in the products of this 
dissemination? Paulette, your paper raises 
interesting challenges for us in this regard. You 
solicited feedback from the tribal grantees, the 
Indian grantees, in your paper. What con- 
stitutes permission? Did they all send back 
something formal that you can document as 
having given their permission to name them in 
your particular manuscript? Jill, you know our 
struggle in preparing the Promising Practices 
Monograph for Circles of Care (Burns & 
Goldman, 1999). What standards should we 
adhere to? What are realistic expectations, 
especially in the face of the demands and the 
lack of resources among our community 
partners to act on these kinds of requests in 
some cases? Who represents the community? 

Is it always the tribal council? Is it the health 
and human services committee? It's often the 
case that it's not clear who represents the 
community. 

Community-based participatory research 
requires an enormous amount of time and 
effort. I'm almost frightened to calculate how 
much time is spent engaging our community 
partners, throughout the research process. The 
tribal review of our manuscripts alone, for 
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example, adds at least seven to nine months, in 
most cases, to submission of a manuscript to a 
journal for review, and if the peer review 
requires substantive modifications, tribal re- 
review is necessary — a process that adds at 
least 12, maybe as much as 18 months of 
capital investment of time and energy. It 
presents a major problem for our younger 
colleagues. 

For those of us in the academy, with many 
younger Native colleagues trying to break 
through the sometimes not-so-transparent 
ceiling in terms of successful promotion and 
tenure in medical schools or colleges of liberal 
arts and sciences, the currency of success is 
grantsmanship and publications. The longer it 
takes, the less likely we are to succeed. There 
are only 11 American Indians and Alaska 
Natives who are associate professors or above 
in medical schools in the entire country. 
When one looks at the rate of success, one 
finds that the vast majority never make it from 
assistant professor to associate professor — and 
that's in large part because we're Native. We 
are assigned a disproportionate amount of 
responsibility for various committees in our 
respective universities. We're asked to teach 
more. The kind of work we do takes two to 
three times longer if we're committed to doing 
it in the way suggested by these papers and in 
which we all believe. So there are many 
challenges before us. One is to ensure that 
people become educated about such issues, 
about possible solutions. And it may have to 
do with modifying routes by which one rises 
through the academy. 

Just two other things to comment on. I've been 
struck by the experiences of two of the tribes 
with which we've been working most closely 
in recent years. Indian Health Service (IHS) 
area IRBs previously performed the review 
process on their behalf. Recently, both tribes 
have rescinded this relationship, concerned 
about tribal sovereignty and delegation of that 
decision-making process to the IHS. This 
action has raised some interesting dilemmas 
for us as we've moved forward in the research. 
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One of the dilemmas has to do with con- 
tracting relationships with the IHS where IHS 
contracts with a given tribe for the tribe to 
provide some of their own health services. The 
IHS asserts — Dr. Bill Freeman, former Director 
of Research for the IHS, was an eloquent 
spokesman in this regard — that if a particular 
facility or group of resources, be they people or 
records, are funded through contract dollars, 
the IHS IRB has jurisdiction. However, if the 
research at issue involves a facility or resources 
funded through a Public Law 93-638 contract 
(Indian Self-Determination and Education 
Assistance Act, 1 975), many tribes believe that 
it's their responsibility, and falls under their 
jurisdiction, not the IHS's. So there's a major 
conflict. Compacting, where tribes choose to 
manage all health service resources, presents 
an entirely different set of circumstances. The 
Indian Health Service readily recognizes that 
the tribes have direct, immediate, and primary 
responsibility for reviewing research that 
involves compacted personnel and facilities. 

The point I'd like to end on, and the biggest 
issue facing us, is this: Our discussion is not 
new. Beginning in the late '60s, many, many 
conferences and symposia have been 
organized around the very same issues and 
concerns voiced here today. Travel forward in 
time to the early '80s — one sees a new series of 
conferences, symposia, and initiatives on 
exactly this same topic. Here we are again, 1 2 
to 1 5 years later. What do we need to do to 
carry forward the momentum that these efforts 
began? To weave these conversations together, 
establishing a sense of continuity and ongoing 
immediacy to the discussion? What does it take 
to move this dialogue further along than we've 
been able to do on past occasions? I don't have 
an answer, but it needs to be one of the critical 
questions we must pose to ourselves if we are 
to have greater impact than history suggests we 
otherwise are likely to have. 

These are my thoughts, then. Thank you very 
much for the opportunity to read your 
respective manuscripts. I look forward to 
future discussion. 
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